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DETAILED ACTION 

Response to Amendment 

1 . This office action is in reply to Applicant's Response dated November 16, 2009. 

2. Claim 1 has been amended and claim 2 was cancelled. 

3. Claims 1, 3-10 are still pending in the present application. 

Response to Arguments 

4. Applicant's arguments with respect to claims 1 -1 0 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Objections 

5. Claim 4 is objected to because of the following informalities. The claim cites as 
follows: 

Method according to Claim 1 , wherein the cost C is dependent on the quadratic 
distances between the signature of the candidate and those of the key images of the 
subset and in that T is the standard deviation of the distribution of the distances of the 

key images of the set from the candidate 

dependent on the distribution of the distances of the key images of the set from the 
candidate. 

However, the last 3 lines do not make sense that the candidate is dependent on 
the distribution of the distances of the key images of the set from the candidate. The 
examiner maintains that the standard deviation T is dependent on the distribution of the 
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distances of the key images of the set from the candidate and therefore interprets it as 
follows: 

Method according to Claim 1 , wherein the cost C is dependent on the quadratic 
distances between the signature of the candidate and those of the key images of the 
subset and in that T is the standard deviation of the distribution of the distances of the 
key images of the set from the candidate and is dependent on the distribution of the 
distances of the key images of the set from the candidate. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or deschbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner In which the Invention was made. 

7. Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Paul E. 
Green (Analyzing Multivariate Data, The Dryden Press, 1978, pp.427-428) in view of 
Pena (Pattern Recognition Letter 20, pp.1 027-1 040) and further in view of Vailaya 
(Technical Report MSU-CPS-96-64) and further in view of Jain et al. (Algorithms for 
Clustering Data, Prentice Hall, 1988, pp.96-101), hereafter referenced as Green and 
Pena and Vailaya and Jain respectively. 
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Regarding claim 1, Green discloses determination (grouped, page 428, line 1-4) 
of a subset (cluster, page 428, line 1-4) from among the set of key images (cluster 
center, page 428, line 1 -4) such that the key images forming the said subset have a 
distance (all object within prespecified threshold are grouped) from the candidate less 
than a threshold T (prespecified distance, page 428, line 1-4) and deletion of the key 
images of the subset to form a new set of key images{Or\ce points enter a cluster they 
are removed from further processing), as disclosed in Sequential Threshold (page 428, 
line 1-4). 

However, Green fails to disclose a method for use in interactive navigation of the video 
sequence comprising: random drawing ofp, from the set of key images by 
automatically extracting shots of interest, p being calculated in such a way as to obtain 
a very good probability of drawing a key image of a prevalent shot, calculation of the 
cost C for each candidate, dependent on the distance from the key images of the set to 
that of the candidate, the distance relating to the signatures, and selection of the 
candidate minimizing the cost C; determination of a seed from among the key images of 
the subset such that it minimizes the cost function C for this subset. 

However, Pena discloses An Empirical Comparison of Four Initialization Methods 
for the K-Means Algorithms. Pena specifically discloses and selecting first seed the 
most centrally located instance (Step 1, Fig.3) in order to initialize seed (Initialization 
method, pp.10, line 11). Even though he selects subsequent seeds differently for 
parallel clustering K clusters, this method cane be used repeatedly in the sequential 
threshold method because once a cluster is formed it is excluded in the next run and a 
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new centroid is calculated. To get the most centrally located instance (instance is 
equivalent to key image), the instance with minimum cost (cost is Sum of distances 
between the candidate and the others, the distance defined by the square error (pp. 
1028, right column, line 9-11) has to be selected. However, it would be computationally 
heavy to calculate the cost for all the instances. One way to reduce the computation is 
to select multiple samples at random and choose the instance with the most centrally 
located one. 

Therefore, given this teaching, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Green by specifically providing 
random drawing of multiple(p) candidates from tlie set of key images, calculation of the 
cost C for each candidate, dependent on the distance from the key images of the set to 
that of the candidate, the distance relating to the signatures, and selection of the 
candidate minimizing the cost C, in order to reduce the computation cost of selecting 
seed for the most centrally located. It is inherent in multiple random sampling, p being 
calculated in such a way as to increase drawing a key image of a prevalent shot, 
because it was well known that statistically multiple sampling increases selecting the 
probability of prevalent shot. 

However, Green and Pena still fails to disclose a method for use in interactive 
navigation of the video sequence; by automatically extracting shots of interest; 
determination of a seed from among the key images of the subset such that it minimizes 
the cost function C for this subset. 
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However, Vailaya discloses Video Clustering. Vailaya specifically discloses a 
method for use in interactive navigation of the video sequence (interactive tools for 
browsing and search image sequences, pp.3, line 38-39) and automatically extracting 
shots of interest (Automatic clustering of travel video shots, pp.2, line 11-15), in order to 
browse and search interactively (pp.3, line 38-39). 

Therefore, given this teaching, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify Bradley and Green by 
specifically providing a method for use in interactive navigation of the video sequence; 
by automatically extracting shots of interest, in order to browse and search 
interactively. The combination of Green and Pena and Vailaya discloses the distance 
fPena; square Error, pp. 1028, right column, line 9-11) defined by the signature 
(Vailaya: Color and Texture features, pp.4, line 5-7 before the bottom). 
However, Green and Pena and Vailaya still fail to disclose determination of a seed from 
among the key images of the subset such that it minimizes the cost function C for this 
subset. 

However, In a yet another same field of endeavor Jain discloses determination 
of a seed from among the key images of the subset such that it minimizes the cost 
function C for this subset (compute new cluster centers as the centroids of the clusters, 
see Algorithm For Iterative Partitional Clustering Step 3 at page 97), in order to 
minimize the square error (pp.96, line 1-4 below chapter 3.3.2). 

Therefore, given this teaching, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Bradley and Green by 
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specifically providing determination of a seed from among the key images of the subset 
such that it minimizes the cost function C for this subset, as taught by Jain, in order to 
minimize the square error. The Green Sequential Threshold Method, incorporating the 
Pena selecting a seed the most centrally located instance, further incorporating the 
Vailaya interactive tools for browsing and search image sequences and automatic 
extraction of shots, further incorporating the Jain computing a new cluster, has all the 
features of claim 1 . It is inherent in the combination of Green and Pena and Vailaya 
and Jain, in order to refine the grouping of key image to those of interest for the 
interactive navigation of the video sequence, because Vailaya discloses interactive tool 
for browsing and searching video sequences as mentioned above . 

Regarding claim 6, the combination of Green and Pena and Vailaya and Jain, as 
applied to claim 1 , discloses shots of a sequence of video images, the sequence being 
split into shots (Vailaya: shot detection, Fig.1 ), shot being represented by one or more 
key images (Vailaya: keyframe extraction, Fig.1), signature or attribute (Vailaya: Color 
and Texture features, pp.4, line 5-7 before the bottom), and comprising a phase of 
partitioning the key images (Vailaya: clustering, Fig.1), on the basis of a comparison 
(Pena: square error, pp. 1028, right column, line 9-11) of the attributes (Vailaya: Color 
and Texture features, pp.4, line 5-7 before the bottom) of the key images, comprising a 
phase of initialization (Pena: selecting centroid) for the selection of at least two key 
images or seeds (keyframe extraction; on the basis of which the comparisons for the 
grouping (Pena: square enror) are performed 
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Regarding claim 7, the combination of Green and Pena and Vailaya and Jain, as 
applied to claim 6, discloses wherein the partitioning phase implements an algorithm of 
the K-means orK-medoid type (Jain: K-means type algorithm at the page 99) by citing 
that K-means type algorithms converge rapidly. 

Regarding claim 8, the combination of Green and Pena and Vailaya and Jain, as 
applied to claim 6, discloses wherein the initialization and partitioning phases are 
iteratively repeated (Green: the process is repeated for the unclustered points, and so 
on, pp.428, line 1 -4), the key images of the most compact cluster obtained in the 
previous iteration being eliminated from the set processed at this previous iteration so 
as to provide a new set on which the new iteration is performed (Green: Once points 
enter a cluster they are removed from further processing, pp.428, line 1-4). 

Regarding claim 10, the combination of Green and Pena and Vailaya and Jain, 
as applied to claim 6, discloses Method of selecting shots of interest (inherent in 
multiple subsampling and Pena: selecting the most centered), these shots being 
prevalent (inherent in multiple sampling) in the video sequence, characterized in that it 
implements implementing the method according to Claim 6, the shots of interest 
corresponding to the grouping performed about the first seed selected (Green: Grouping 
a cluster in a prespecified distance, page 428, line 1-4). 

8. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Green and Pena and Vailaya and Jain in view of Palmer ("Density 
Biased Sampling: An Improved Method for Data Mining and Clustering", Proceedings of 
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ACM SIGMOD International Conference on Management of Data, 2000), hereafter 
referenced Palmer. 

Regarding claim 3, the combination of Green and Pena and Vailaya and Jain 
discloses everything claimed as applied before (see claim 1), however it fails to disclose 
wherein the key images are weighted, as regards their signature, as a function of the 
length of the shots of the video sequence that they characterize and in that the random 
draw is biased by the weight of the key images. However the examiner maintains that it 
was well known in the art to provide the above as taught by Palmer. 

In a similar field of endeavor Palmer discloses wherein the key images are 
weighted, as regards their signature, as a function of the length of the shots of the video 
sequence that they characterize and in that the random draw is biased by the weight of 
the key images ((iii)the sampling is biased by group size, page 3), in order to improve 
on uniform sampling (pp.2, right col., line 30-33). 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify the combination of Green and Pena and Vailaya 
and Jain by providing the biased sampling, as taught by Palmer, in order to improve on 
uniform sampling. The Green Sequential Threshold Method, incorporating the Pena 
selecting a seed the most centrally located instance, further incorporating the Vailaya 
interactive tools for browsing and search image sequences and automatic extraction of 
shots, further incorporating the Jain computing a new cluster, further incorporating the 
Palmer biased sampling by group size, has all the features of claim 3. 
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9. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Green and Pena and Vailaya and Jain in view of Foote (US patent 
6,774,917), hereafter referenced as Foote. 

Regarding claim 4, the combination of Green and Pena and Vailaya and Jain 
discloses everything claimed as applied before (see claim 1). Bradley further discloses 
wherein the cost C is dependent on the quadratic distances (the sum of squared 
distances of each data point to nearest means) between the signature of the candidate 
and those of the key images of the subset. However it fails to disclose that T is the 
standard deviation of the distribution of the distances of the l<ey images of the set from 
the candidate and dependent on the distribution of the distance of the key images of the 
set from the candidates. 

In a similar field of endeavor Foote discloses Methods and Apparatus for 
Interactive similarity searching, Retrieval, and Browsing video. Specifically Foote 
discloses that a multiple of standard deviation is used as threshold(T) in the figure 18 
and the column 5, line 36-39, for the purpose of detecting class membership effectively 
using statistics (col. 15, line 66 - col. 16, line 1-5). 

Therefore It would have been obvious to one of ordinary person in the art at the 
time the Invention was made to modify the combination of Green and Pena and Vailaya 
and Jain by providing 7 as the standard deviation of the distribution of the distances of 
the key images of the set from the candidate and dependent on the distribution 
(inherent in standard deviation) of the distance of the key images of the set from the 
candidates, as taught by Foote, for the purpose of detecting class membership 
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effectively using statistics. The Green Sequential Threshold Method, incorporating the 
Pena selecting a seed the most centrally located instance, further incorporating the 
Vailaya interactive tools for browsing and search image sequences and automatic 
extraction of shots, further incorporating the Jain computing a new cluster, further 
incorporating the Foote setting a multiple of standard deviation as threshold(T), has all 
the features of claim 4. 

1 0. Claim 5 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over the 
combination of Green and Pena and Vailaya and Jain in view of Wan et al. ("A 
multiresolution color clustering approach to image indexing and retrieval", Proceedings 
of 1998 IEEE International conference, pp.3705-3708), hereafter referenced as Wan. 

Regarding claim 5, the combination of Green and Pena and Vailaya and Jain 
discloses everything claimed as applied before (see claim 1), however it fails to disclose 
wherein the signature of an image relates to the dominant color, in order to speed up 
the retrieval (pp. 3706, line, right col., 6-7) 

In a similar field of endeavor Wan discloses the dominant color as a feature (sub 
clause 3.1 , pp.3706, right col., line 14-23), which is equivalent to signature of an image 
relates to the dominate color. 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify the combination of Green and Pena and Vailaya 
and Jain by providing the dominant color as a feature, as taught by Wan, in order to 
speed up the retrieval. The Green Sequential Threshold Method, incorporating the Pena 
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selecting a seed tlie most centrally located instance, further incorporating the Vailaya 
interactive tools for browsing and search image sequences and automatic extraction of 
shots, further incorporating the Jain computing a new cluster, further incorporating the 
Wan setting the dominant color as a feature, has all the features of claim 5. 

1 1 . Claim 9 Is rejected under 35 U.S.C. 1 03(a) as being unpatentable over the 
combination of Green and Pena and Vailaya and Jain in view of Turi et al. (Proceedings 
of the LASTED international Conference Signal and Image Processing, Oct 27- 
31,1 998, pp.345-349), hereafter referenced as Turi. 

Regarding claim 9, the combination of Green and Pena and Vailaya and Jain 
disclose everything as claimed as above (see claim 8). However the combination of 
Green and Pena and Vailaya and Jain fails to disclose wherein the stopping criterion for 
the iterations is dependent on the number ofl<ey images not belonging to the most 
compact cluster selected or else is dependent on the averages of the intra-cluster 
distances, 

In a similar field of endeavor Turi discloses K-means Clustering for Colour Image 
Segmentation with Detection of K. Turi specifically discloses the stopping criterion 
dependent on the averages of the intra-cluster distances (Equation 1 at the page 346), 
in order to make the process entirely automated (pp. 345, 3 lines below chapter 2). 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify the combination of Green and Pena and Vailaya 
and Jain by providing the average intra-cluster distance as a stopping criterion, as 
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taught by Turi, in order to nnal<e tine process entirely automated. The Green Sequential 
Threshold Method, incorporating the Pena selecting a seed the most centrally located 
instance, further incorporating the Vailaya interactive tools for browsing and search 
image sequences and automatic extraction of shots, further incorporating the Jain 
computing a new cluster, further incorporating the Turi Stopping Criterion as the 
average intra-cluster distance, has all the features of claim 9. 



Conclusion 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to HEE-YONG KIM whose telephone number is (571)270- 
3669. The examiner can normally be reached on Monday-Thursday, 8:00am-5pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha Banks-Harold can be reached on 571-272-7905. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Examiner, Art Unit 2621 
/Marsha D. Banks-Harold/ 

Supervisory Patent Examiner, Art Unit 2621 

/Andy S. Rao/ 

Primary Examiner, Art Unit 2621 
March 23, 2010 



